Summary

Previously, the expected and actual messages columns of the comparison csv generated by the session compare CLI when the compared session does not match the expected results did not always contain helpful details and often cells were too long. Specifically, extra messages would throw off the comparison and data in a single cell could go over the csv line limit. Now, three enhancements make the output more usable:

• The comparison is more robust and handles extra messages without throwing off other comparisons
• Multiple duplicate normalized messages are now collapsed into a single entry with an indicator of how many there are.
• by default, only different messages, meaning messages that weren't present in the expected but were in the actual and vice-versa are displayed. There is a new flag on both the session compare and execute_script CLI commands to revert this behavior.

Testing Guidance

To see this in practice, manually edit a messages element in an expected results json of an execution script and run the script to see the differences. Here's one example:

1. Edit execution_scripts/validator/validator_suite_with_file_input_expected.json by replacing the messages element of the result entry with test_id value "dev_validator-patient_group-patient_validate_test" with the following

        [
           {
               "message": "ADDED",
               "type": "warning"
           },
           {
               "message": "Patient/85: Patient.extension[0].extension[0].value.ofType(Coding): The definition for the Code System with URI 'urn:oid:2.16.840.1.113883.6.238' doesn't provide any codes so the code cannot be validated",
               "type": "info"
           },
           {
               "message": "ADDED",
               "type": "warning"
           },
           {
               "message": "Patient/XX: Patient.extension[1].extension[0].value.ofType(Coding): The definition for the Code System with URI 'urn:oid:2.16.840.1.113883.6.238' doesn't provide any codes so the code cannot be validated",
               "type": "info"
           },
           {
               "message": "Patient/85: Patient.identifier[2].type: None of the codings provided are in the value set 'IdentifierType' (http://hl7.org/fhir/ValueSet/identifier-type|4.0.1), and a coding should come from this value set unless it has no suitable code (note that the validator cannot judge what is suitable) (codes = http://terminology.hl7.org/CodeSystem/v2-0203#SS)",
               "type": "warning"
           },
           {
               "message": "ADDED",
               "type": "warning"
           }
       ]

2. Start inferno services and the test kit
3. Execute the script corresponding to the updated results file bundle exec inferno execute_script execution_scripts/validator/validator_suite_with_file_input.yaml
4. It should result in a different result and generate the csv comparison file in the same directory as the expected results file.
5. Examine the csv comparison file and confirm that

• The 3 "Added" messages are collapsed into a single row in the expected column (weren't present in the actual)
• The only other message entries is the line with the patient id changed from 85 to XX - the other rows with the patient ids left the same will not be present.

6. Run the session compare CLI with the option to include only different messages off: find the session id in the output of the execute script command and run bundle exec inferno session compare <session id> -m -r -f execution_scripts/validator/validator_suite_with_file_input_expected.json
7. This should result in a different result and generate the csv comparison file in the same directory as the expected results file.
8. Examine the csv comparison file and confirm that it is the same as above, but with entries for the rows where the patient id wasn't changed in the expected file.